Essential conditions for evolution of communication within a species 
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A major obstacle in analyzing the evolution of information exchange and processing is our insuf- 
ficient understanding of the underlying signaling and decision-making biological mechanisms. For 
instance, it is unclear why are humans unique in developing such extensive communication abilities. 
To treat this problem, a method based on the mutual information approach is developed that evalu- 
ates the information content of communication between interacting individuals through correlations 
of their behavior patterns (rather than calculating the information load of exchanged discrete sig- 
nals, e.g. Shannon entropy). It predicts that correlated interactions of the indirect reciprocity type 
together with affective behavior and selection rules changing with time are necessary conditions for 
the emergence of significant information exchange. Population size variations accelerate this devel- 
opment. These results are supported by evidence of demographic bottlenecks, distinguishing human 
from other species' (e.g. apes) evolution line. They indicate as well new pathways for evolution of 
information based phenomena, such as intelligence and complexity. 

PACS numbers: 



Information is a resource, like energy or food, con- 
tributing to the evolutionary success of a population and 
its members. As a consequence, the majority of biological 
species, from bacteria and plants to human beings, have 
developed skills to sense their environment, communicate 
with each other [l|, Q, H, H| and in some cases even inter- 
fere with the signalling of hostile organisms Q . The abili- 
ties to acquire and process information, however, change 
significantly across populations. Humans, for instance, 
demonstrate unique techniques of information transmis- 
sion [6( . The variety of communication skills and relative 
brain sizes Q indicates important constraints on the de- 
velopment of human-like information exchange and pro- 
cessing, which provide superior abilities for joining efforts 
to control and modify the environment. 

Two main factors constrain the evolution of informa- 
tion exchange and intelligent processing. First, informa- 
tion comes at a cost that can be higher than the cor- 
responding evolutionary gain. Many organisms there- 
fore, avoid sustaining excessive sensing and decision mak- 
ing abilities by preferring, for example, stochastic (non- 
responsive) behavior in a fluctuating environment [3 Q ■ 
Second, individual fitness can be improved in the course 
of a competition by selfish hiding of useful information, 
making cooperative fair signaling evolutionary disadvan- 
tageous. Mimicry, deception [I0( and other methods of 
selfish information protection are common in the living 
world, from populations of bacteria [ll[ to states of gov- 
ernment. It is however still unclear what allowed humans 
to overcome them with such an efficiency. 

We address here the question: What are the necessary 
evolutionary selection conditions for the development 
of communication (significant information exchange and 
processing) between members of a population? Previ- 
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ous attempts to answer similar questions, can be roughly 
separated into two main categories: First, analysis of the 
emergence of various communicative behavior strategies, 
e.g. cooperation 0113, fair signaling [jl 
quisition of a common language [l6|, [Tt], IbSl \Vt 
Second, applying information theory to investigate how 
signals become associated with meaning [22| . In this 
work the second approach is followed. The amount of 
exchanged information is evaluated by the strength of 
correlations between different signals. This is different 
from the conventional approach that quantifies only the 
information content of discrete signals. 

Consider interacting organisms generating and display- 
ing some signals, such as information carrying messages 
or behavior and display patterns. How can one be sure 
that they communicate, in the sense that some informa- 
tion is transmitted, comprehended and utilized for choos- 
ing a specific behavior, e.g. cooperation or selfishness? 
For instance, no communication is required for interac- 
tions where the competitors choose their behavior inde- 
pendently of each other. On the other hand, significant 
information exchange occurs in the course of correlated 
reactions between two interacting organisms, that are 
committed to a common role agreement [231 ■ e.g. who 
is the donor and who is the recipient. Consequently, cor- 
relations between observable behavior patterns can serve 
as a quantitative measure of information exchange, al- 
lowing to construct an ab initio model of communication 
with no a priori assumptions concerning specific mecha- 
nisms of information transfer and processing. 

In this paper, we demonstrate that a unique set of evo- 
lutionary conditions provides a solution both for a sta- 
ble, maximal information exchange between individuals 
and for the naturally occurring diversity of communica- 
tion abilities across populations. Communication skills 
in populations are evaluated by presence of correlated, 
rather than independent, behavior patterns. Correlated 
behavior is predicted to co-evolve with an ability to af- 
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feet the competitor's behavior, creating populations with 
a mixture of evolutionary important correlated interac- 
tions of the indirect reciprocity type [HI: and more fre- 
quent non-responsive behavior, lacking any evolutionary 
gain. Another result is that fluctuations in population 
size, which are extreme in human evolution compared to 
other great apes species [25[, are an indicator of faster 
evolution of information exchange and processing. The 
supporting analytical tools are presented in the supple- 
mentary material (SM) section. 

Any evolvable individual property (a phenotype) 
emerges as a mutation in a population, subsequently de- 
veloping or waning with generations as a consequence of 
natural selection. The selection process can be approxi- 
mated by a game [2(| [U HI] where players A and B react 
mutually. W pq will be associated with the payoff that a 
player receives for his reaction p vs. q of the competitor. 
In a given interaction, each competitor may obtain pay- 
offs according to different weight tables. This allows one 
to take into account that an individual may be found in 
different modes (states such as e.g. age, intra-population 
rank, sex, ability to acquire information, etc.) that influ- 
ence his payoff (see Fig. 1A). Some of the payoff tables 
have been named by analogy to real-life situations (e.g. 
Prisoner's Dilemma, Chicken, Battle of the Sexes etc.). 
It must be stressed though that in these cases both com- 
petitors play under identical weight tables (see Fig. IB). 

Consider repeated interactions between two individu- 
als A and B. Let us assume that each interaction in- 
cludes exchange of some information carrying messages 
of any type, together with either selfish S or cooperation 
C reactions. The terms "reaction" or "response", rather 
than strategy, are used to emphasize that individuals do 
not act independently, but may react to their opponent's 
actions. The term "strategy" refers here to the deci- 
sion process that leads to a specific act. For instance, in 
this context one may say that a player has used the Tit- 
for-Tat strategy to choose either cooperative or selfish 
reaction, while playing the Prisoner's Dilemma game. 

This paper demonstrates that only the statistics of mu- 
tual reactions is required to estimate the amount of infor- 
mation that A and B exchange, comprehend and utilize 
during their strategic decision making. An alternative 
approach could be based on analysis of information load 
of messages generated by the competitors. There ex- 
ist numerous measures for information in language-like 
signals, including Shannon entropy [29j and Kolmogorov 
complexity [3C| . Association of information in a signal 
with communication, implicitly assumes that competi- 
tors fully understand each message. This assumption, 
however, is not applicable to evolvable systems, since de- 
veloping signals are always ambiguous. 

All information exchanged by competitors A and B, 
pertinent to their mutual behavior, will be associated to 
the amount of information that one can infer concerning 
the unknown behavior of A from observing the behav- 
ior of B (or vice versa). For instance, an external ob- 
server, by observing during an interaction the reaction of 



B, e.g. cooperation, will be able to predict with reduced 
uncertainty (equals to information gain) the reaction of 
A. Competitors lacking any mutual knowledge, have to 
exchange information during their interaction, in order 
to achieve dependent, rather than random, mutual be- 
havior. 

Fortunately, information theory provides a standard 
expression for mutual information [3l| , describing reduc- 
tion in uncertainty of one signal through the knowledge 
of another. So far it was applied to analysis of corre- 
lations, complexity and communication in physical [32| . 
linguistic [33| and biological systems [34], [3g, [3(1 [37| • In 
our case, mutual information / indicates how much infor- 
mation (in bits) can be derived from the reaction of one 
of the competitors, while knowing certainly the reaction 
of the other: 

1 = p,5,o ^ l0g2 ((n P s + n P c)(n Sq + n Cq ))} ' (1) 

where Q pq is the probability of reactions p against q. This 
work derives the selection rules that maximize the value 
of mutual information I in an evolving system, adopting 
it as a measure of communication. 

The properties of the mutual information expression 
([T]) justify its relevance to the problem of communica- 
tion. Mutual information / varies from to 1 bit, reach- 
ing its maximum in case of positively (S vs. S and C 
vs. C, flss = ^cc = 0.5 and fl S c = ^cs = 0) or 
negatively (S vs. C and C vs. S, Qsc — ^cs = 0.5 
and Qss = ^cc = 0) correlated reactions, indicat- 
ing that they require significant information exchange. 
The minimum values of / = correspond to indepen- 
dent mutual reactions (as in case of mixed strategies) 
(S is generated with probability xs, ^ss = ^Ij^sc = 
zs(l ~ x s ), ties = (1 - x s )x s , Qcc = (1 - x s ) 2 ), in- 
cluding unconditional selfishness xs = 1 (S vs. S, 
flss = l,^sc = Qcs — ^cc — 0) and unconditional 
cooperation xs = (C vs. C, flee — T^sc = ^cs = 
^55 — 0). Independent reactions, indeed, do not re- 
quire any information exchange. Analysis of the evolu- 
tionary stability of populations with correlated reactions 
is therefore of major importance to describe the evolution 
of abilities to exchange and process information. 

In this model, an individual A possesses several behav- 
ior modes (states) k e {1, N}, each one characterized by 
three evolvable parameters (e^, ct^Pk) an d weight table 
W pq . In the course of a competition, A acts in a spe- 
cific mode k with probability e^5, generates either selfish 
S or cooperative C reactions with a strategy defined by 
( a k'Pk) an d receives payoffs according to W^ q (Fig. 2A). 
The conditional probabilities (a£ , /3^) define the statis- 
tics of reactions in each behavior mode: ak and 1 — 
are, respectively, the probabilities to choose cooperation 
C and selfish S reactions against the competitor's uncon- 
ditional selfish reaction S, while fik and 1 — [3k are the 
probabilities to choose cooperation C and selfish S reac- 
tions against the competitor's unconditional cooperation 
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reaction C (see Fig. 2B). In this context, (a£,0^) can 
be viewed as selfishness aversion and cooperation attrac- 
tion correspondingly. Affective behavior is introduced 
by allowing the phenotypes of competing individuals to 
depend on each other. 

The "mode" index makes possible a multi-game/multi- 
strategy generalization, taking into account that individ- 
uals may possess several behavior states (e.g. age, intra- 
population rank, sex, ability to acquire information, etc.) 
that affect both the payoffs W* and behavior (a^,0f) 
of an individual A during an interaction. Each mode, 
therefore, corresponds to a specific strategy and payoff 
table (see Fig 2A). For instance, when A in mode(A) re- 
acts with S vs. reaction C of B in mode (B), he receives 



a payoff W, 



mode(A) 
SC 



while _B's payoff will be W. 



mode(B) 



CS 



shown in Fig 1A. All individuals are assumed to share 
the same set of behavior modes fee 1, N. 

This phenotype description follows with some alterna- 
tions previous attempts to incorporate conditional cor- 
relations into game theory and biological applications 
@, H 11 HI El IH, EI H EH . Conditional probabili- 
ties a and are closely related to the reactive strategies 
approach [H E3 • Reactive strategies denote the proba- 
bilities to cooperate after the competitor has cooperated 
or defected during the previous interaction, while con- 
ditional probabilities assign probabilities for cooperation 
during the same interaction. This approach makes possi- 
ble a general description of highly communicative game 
strategies for choosing a specific reaction (like Tit-for- 
Tat), together with non-communicative ones (e.g. mixed 
strategies). Any game strategy in a course of a com- 
petition leads to a specific statistics of mutual reactions 
flpq. Conditional probabilities allow describing all possi- 
ble statistics of reaction, effectively taking into account 
any game strategy. The individual payoff, defining the 
evolutionary success, can be expressed though a and 0, 
since it depends solely on the statistics tt pq . 

The payoff F(A, B) of an individual A interacting with 
individual B is: 



F(A, 



^cc^cc 



where the summation includes all possible interactions 
(mode i of A vs. mode j of B), taking into account the 
corresponding probabilities of such interaction efef and 
the statistics of the mutual reactions £1^ . To calculate 
the payoff F(A, B) and the information exchange rate I, 
one should derive the statistics of the mutual reactions 
£l p ^ as a function of the competing individuals' pheno- 
types. 

The statistics of the mutual reactions Q p ^ during an 
interaction of individuals A in mode i vs. B in mode j, is 
a function of the phenotypes (oti,0i), (atj,0j) and their 
reaction order in the course of a competition (the first to 
respond lacks confident information on the competitor's 
intentions). The value of f2*'| follows from definitions of 



a and 0, e.g.: O^fo, the probability of cooperation vs. 
selfishness in a competition is equal to the conditional 
probability a, of A to cooperate with a selfish competitor, 
multiplied by the unconditional probability jji of B to 
provide reaction S during a competition with A (see Fig. 
2B): 



Qss = (1 - otifai, fl C c =0i(l- 7,-i) 
Has = «i7ii! ^sc = (1 - AX 1 - Jji), 



(3) 



where 



7ji 1S: 



jji(aj,0j,ai,0i) 



(l_^)_(l_ A )( a ._^.) 



(4) 



Derivation of eq. (U|) follows from symmetry consider- 
ations, for instance the statistics 0^ and fE'g have to 
be identical. Fortunately, fl p ^ is independent of the or- 
der of reactions in interactions which contribute to the 
development of communication (see SM IA 3 b[ IA 41 and 
IX5i 

Two individuals possessing the same behavior mode 
(a.i,0i) = [a.j,0j) = {a, 0) exchange during an interac- 
tion an amount of information: 



I{a, 0) = log 2 



(I - O^ 1 ""^! - 0)O--m-t)g°<100{l-l) 

7 7 (1"7) 7 



where, according to eq. ©, 7 = = j) is: 
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1-/3 
1 + a- 0' 



(6) 



Eq. (O follows from Jl}, taking into account eqs. © and 
dU) (see SM IA3cp . Expression §5§ provides a quantita- 
tive measure of the individual's information processing 
capability in each behavior mode (a, 0). 

According to eq. ([5]), the behavior modes (a — 1,0 — 
0) and (a = 0, = 1) possess the maximal informa- 
tion exchange rate (1 bit per interaction), as shown in 
Fig. 2C. These modes can be classified according the 
(2) statistics of reactions il pq they generate. The mode 
(a = 0, = 1) corresponds to positively correlated re- 
actions (tl S s = 0.5, Vt C c = 0.5, ftsc = ^cs = 0), 
similar to synchronous interactions of cells in a multi- 
cellular organism, e.g. heart contraction. The mode 
(a = 1,0 = 0) corresponds to negatively correlated 
statistics (Use = 0.5, Qcs = 0.5, Qss = ^cc = 0), such 
as interactions of the indirect reciprocity, taking C as 
donor and S as recipient). The flux of information is zero 
for the modes (a = 0), (a = 0,0) and (a,0 — 1), corre- 
sponding to random mutual reactions, unconditional self- 
ishness (Qss — 1) ^sc — ^cs — Qcc = 0) and uncondi- 
tional cooperation (flee = l,^sc = ^les = ^Iss = 0). 

The evolution of positively correlated reactions (a — 
0,0 = 1) requires much longer times than the evolution 
of negatively correlated ones (a = 1,0 = 0). This is a 
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consequence of the slow evolution near each of the the 
axes a — and = 1, since the interactions in the popu- 
lations along these axes are all of the unconditional self- 
ishness type (Oss = 1) along a = or cooperation type 
(f^cc = 1) along 0=1. There is no evolutionary drive to 
move between different populations possessing the same 
statistics of reactions, since they correspond to the same 
fitness value ©. Further analysis, therefore, will focus 
on the development of communication abilities based on 
negatively correlated reactions (a = 1, = 0), leading to 
interesting analogies with real world phenomena. 

A population consisting of phenotypes with a single 
negatively correlated behavior mode (a — 1,0 = 0), is 
evolutionary unstable under any weight table W pq . This 
can be demonstrated as follows: an individual of this type 
acts against an identical competitor either as a donor or 
a recipient with equal probabilities {Slsc = 0.5, flcs = 
0.5, Qss = Slcc = 0), receiving an average payoff of 
{Wsc + W / cs)/2- He demonstrates, however, uncondi- 
tional cooperation (flcs ~ lj^sc ~ ^ss ~ ficc ~ 0) 
or selfishness {Qsc ~ 1>^CS ~ ^ss ~ ^cc ~ 0) 



weight for each competition: 



W H cx 



l-e B 





c 


S 


c 


Wcc 


Wcs 


s 


W sc 


W ss 



(8) 



against mutants described by (otj 



1 



Aa, 0j 



0) and by (a,- = l,/3j = A0) Aa, A/3 < 0, pro- 
viding them payoffs Wsc and Wcs- These abrupt 
changes of behavior are a consequence of the instability 
of 7ji(aj, 0j,ai, 0i) ([3]) near the point (a — 1,0 = 0): 
7^(1,0,1,0) = 0.5, lim Ac ^o 7^(1, 0,1 - Aa,0) = 
and limA/3^0 7ji(lj 0, 1, A/3) = 1. Either of the mu- 



The interactions of a population with high affect abili- 
ties e — > 1 are composed, therefore, of a small amount (cx 
1 — e) of evolutionary important competitions with nega- 
tively correlated reactions and significant information ex- 
change, together with more frequent non-responsive be- 
havior lacking any evolutionary gain (Fig. 3). 

In such a population, communication abilities will 
evolve when W pq (see |S]) fluctuate around the values cor- 
responding to games of Chicken, Leader and Battle of 
the Sexes (see Fig. IB and SM IA 6[) . No development 
occurs either when W pq do not fluctuate or correspond 
to other games, e.g. Prisoner's Dilemma. To describe 
the evolutionary dynamics, the instantaneous state of a 
population is defined as a density distribution over the 
phenotype space p{r) = (a,0,e). The time develop- 
ment is governed by the replicator dynamics equations 
[46| : it consists of the emergence of mutations due to a 
diffusion-like process and redistribution between the old 
phenotypes according to natural selection: 



dp{~r) 
dt 



= p(~r) 



=l+D 



Tgen \ F \ 



d 2 P 

d'r'- 



,F = I p(?)F(?)d 
(9) 



tants (a, = 1 — Aa,0j = 0) and (ay = l,0j — A/3), where F denotes the fitness, F the average fitness, D the 



therefore, outperforms the (a = 1,0 = 0) host, since 
W sc > {Wsc + Wcs)/ 1 * or W CS > {W SC + W CS )/2. 
Consequently, evolutionary stability of negative correla- 
tions requires phenotypes composed of at least two dif- 
ferent behavior modes. 

An example of a population with two behavior modes 
possessing evolutionary stability and diversity of com- 
munication skills can be constructed as follows (SM 
IA3el and IA 4ft . The phenotype of an individual A 
will include two behavior modes (responsive (i?) and 
non-responsive {NR)), described by three parameters 
{a. a, 0a, £a)i where takes now the meaning of affect 
ability. In the course of an interaction, the individuals A 
and B possess non-responsive behavior mode {NR) with 
probabilities defined by the affect abilities of the com- 
petitor, meaning that non-responsive behavior is not a 
personal choice, but enforced by the opponent. In case 
of A, therefore, 1 — es and are the probabilities to 
act in responsive (i?) and non-responsive {NR) modes 
correspondingly (Fig. 3). In the non- responsive mode, 
an individual behaves stochastically, generating reactions 
with probabilities matching the average statistics of its 
reactions, and receives no payoff: 



diffusion coefficient of the mutation process and T gen the 
time-span of a single generation. These equations can 
be reduced to the local velocities of a population in the 
phenotype space fSM IA 2[) . In the case of a particle- like 
population confined to the vicinity of its average pheno- 
type one obtains, taking condition ([7]) into account: 
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(7) 



In the responsive mode, its behavior is defined by the 
conditional probabilities {a,0), and, therefore, depends 
on the reactions of the competitor. The lower the num- 
ber of pairwise interactions in this mode, the greater the 



t t r (11) 

where |Aa| 2 , \A0\ 2 and |Ae| 2 are the spreads of the pop- 
ulation along a, and e directions correspondingly. The 
development of affect abilities e depends on the changes 
in a and (v e oc \{v a , vp)\). Populations, therefore, can 
possess many different values of affect abilities {v e = 
for any value of e) and corresponding information skills 
at the stable points v a = vp = 0, matching the diversity 
of communication abilities in nature (see Figs. 4A-4D). 

The stronger the fluctuations of the size of a popula- 
tion, the faster the development time of communication 
abilities in a high affect, negatively correlated behavior 
mode: (e 1, a = 1, = 0) (Fig. 4E): 
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since fluctuations of the payoffs W pq correspond to fluc- 
tuations in the size S of a population. The time depen- 
dent evolutionary conditions keep the population away 
from its stable points (v a = vp = v e = 0), leading 
to an increase of the affect abilities (v e > 0) and evo- 
lutionary stability of the maximum possible informa- 
tion exchange in the indirect reciprocity behavior mode 
(e— >l,a = l,/3 = 0). Chicken and Battle of the Sexes 
games possess the fastest development rate of communi- 
cation abilities (see Fig. 4F). 

The prediction that development of communication is 
accelerated with population size variations, is supported 
by evidence of significant demographic fluctuations dur- 
ing human evolution (25J. In addition, the long time re- 
quired to develop multicellularity on Earth (about 75% 
of the entire evolution time-span) HH can be as- 
sociated with the slow development of positively corre- 
lated synchronous behavior mode (a = 0, j3 = 1). Nega- 
tively correlated reactions of the indirect reciprocity type 
(a = = 0) lack selfish competitions (Clss = 0) and 
match, therefore, the observations of relatively low ag- 
gression levels in humans [49j . From a single interaction's 
perspective, indirect reciprocity can be interpreted as al- 
truistic. It allows one to suggest that non-kin altruism, 
considered to be a unique human property , could be 
a by-product of evolution of communication abilities. 

The derived conditions are essential for evolution and 
diversity of communication ( - SM IA3"e|) . Deviations from 
the scaling of the payoffs per behavior mode ([5]) (e.g. 
scaling per pairwise interactions) destroy the evolution- 
ary stability of the high affect, negatively correlated 
(e — » 1, a = 1, (3 = 0) behavior mode. Non-responsive be- 
havior with finite, rather than zero ([7]), payoffs prevents 
multi-stability (diversity) of the communication develop- 
ment. The inability to affect a competitor to behave in 
a specific behavior mode makes evolution of the infor- 
mation exchange and processing impossible. The same 
statements are valid in the case of multi-behavior mode 
phenotypes. Non-diverse (all populations converge to the 
same state) development of negative correlations, how- 
ever, is possible, raising the question whether there exists 
a scheme for evolution of information exchange, which is 
different from the emergence of human-like communica- 
tion abilities? 

The results of this work indicate that development of 



artificial life or intelligence may be impossible without 
selection rules changing with time. The attempts to cre- 
ate artificial evolution of competing digital organisms led 
so far to a limited, rather than unbounded, growth of 
complexity (5ll . |52| . Evolution of communication can be 
considered as a base for significant complexity growth 
in living and artificial systems. The analogies of these 
results with human evolution (e.g. presence of demo- 
graphic bottlenecks) indicate a promising direction for 
the artificial evolution of intelligent systems composed of 
communicating agents. 

The developed framework does not predict exact mech- 
anism for the evolution of communication. Only general 
evolutionary conditions are derived that lead to corre- 
lated statistics of reactions, assuming that the mecha- 
nisms for the necessary information exchange and pro- 
cessing can be implemented in various ways and are 
species' dependent. 

To conclude, a mathematical framework based on ob- 
servable, correlated behavior patterns was developed for 
analysis of evolution of information exchange in popu- 
lations. It predicts that the communication abilities of 
populations are related to negatively correlated interac- 
tions (such as donor vs. recipient) and affective behavior. 
It was found that fluctuations in population size acceler- 
ate significantly the evolution of these abilities, allowing 
thus to explain the uniqueness of the human-line evo- 
lution. Competitions similar to games of Chicken and 
Battle of the Sexes lead to a faster evolution of commu- 
nication. An experimental verification of models based 
on this framework is in principle possible, since the de- 
veloped analytical tools are based on observable param- 
eters, such as statistics of mutual reactions during com- 
petitions, rather than exact modeling of information ex- 
change and decision making biological mechanisms. 
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FIG. 1: Evolution as a game. (A) Interaction of individuals A and B, possessing behavior modes modef^) and mode(B) 
correspondingly. A and B generate reactions p and q, receiving payoffs according to tables specific to each mode, e.g. W^, ode< - A ^ 
is the payoff of A for its reaction p vs. reaction q of its competitor. Indices p and q represent either selfish S or cooperative 
C reactions. Different modes correspond to the possible states of the competitors (e.g. age, intra-population rank, sex, ability 
to acquire information and etc. . . . ) that determine the individual payoffs Wp q during an interaction. All individuals are 
assumed to share the same set of modes k G {1, iV}. (B) Classification of payoff values (6, c) according to evolutionary game 
theoretical models, assuming that the competing individuals are in the same mode, with payoffs Wss = 0, Wsc = b, Wcs = c 
and Wcc = I- Payoff values within the unmarked areas lack specific names. 
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FIG. 2: Correlations of mutual reactions and information exchange in the course of a competition. (A) The strategy of an 
individual A in behavior mode i is defined by ai and a denotes the conditional probability of player's A cooperation 
reaction C vs. the selfish reaction S of its competitor B, whereas j3 denotes the conditional probability for cooperation C vs. 
cooperation C of the competitor. Selfishness aversion and cooperation attraction are possible interpretations for a and f3. (B) 
The statistics fij* for reaction p by A mode{i) vs. q by B modeU) follows from the definitions of a and /3 (eq. ©). To determine 
the probability of C vs. S interaction, for instance, one must multiply the conditional probability at of A to cooperate 
with a selfish competitor B by the unconditional probability jji of B to provide reaction S during a competition with A. The 
values of unconditional probabilities jji and 7y follow from symmetry considerations (eq. Q). (C) Contour plot of information 
exchange per interaction I(a,f3) in populations consisting of individuals competing in the same mode (a, 13). The diagonal 
(a = j3) together with the axes (a — 0) and (f3 = 1) correspond to a minimum of bits exchanged per interaction matching the 
cases of random mutual reactions, of unconditional cooperation (/3 = l|ficc = 1> Qsc ~ Qcs = &ss ~ 0) and of unconditional 
selfishness (a = 0|f2ss = 1, Qsc ~ Qcs = Qcc = 0). A maximum of 1 bit per interaction is achieved at the points (a — 0, f3 — 
1) and (a — 1,(3 — 0) (marked with a (^)), corresponding to the synchronous (Qss = 0.5, flee = 0.5, flsc = ^cs = 0) and 
indirect reciprocity (flsc = 0.5, Qcs = 0.5, flss = ^cc = 0) behavior modes. Tiny deviations from the synchronous point 
(a = 0, j3 = 1) will bring the mutual information exchange to 0, due to the very steep gradient of I(a, (3) in this region. 
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FIG. 3: Essential conditions for evolution and diversity of communication in a population with two behavior modes per 
individual. An individual A is either in a non-responsive or a responsive mode, with the probabilities es and 1 — es, defined by 
the affect ability e_g of its competitor B. The affect ability e may be considered as an individual's skill to influence the behavior 
of its competitor, enforcing him in this way to behave stochastically. There is zero payoff for non-responsive behavior (W pq = 0). 
The lower the probability to be in responsive mode, the greater the payoff during a competition. In a population with developed 
affect abilities (e — > 1), the responsive behavior corresponds to high evolutionary payoffs, but interactions of this type are rare. 
In the non-responsive mode an individual generates arbitrary reaction with probabilities matching the average statistics of 
its reactions. Non-responsive behavior can be interpreted as a reaction induced by a signal from an arbitrary member of the 
population, rather than from a competitor. The reaction in this case matches the statistics of the mean reactions, since the 
signal effectively averages over all player's possible interactions within the population. 
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FIG. 4: Development of communication abilities as a consequence of fluctuations in population size. (A) For the population 
presented in Fig. 3., the local velocity v(e, a, j3, b, c) = (v a ,Vp,v e ) defines the instantaneous evolution of a confined population 
in the phenotype space (a, /3, e). Development of affect abilities v f = de/t is proportional to v a and vp, the changes with time of 
selfishness aversion a and cooperation attraction (3 correspondingly. (B) The vector field of v in case of (e = 0.5, b = 1.5, c = 0.5). 
A population converges to one of its stable states with v a = vp = v e — 0. Lines and points of convergence are marked in 
bold. (C) The stable points (a s t,/3st) in the case e — 0.5, as a function of the payoffs (6, c). (D) Convergence of a population 
towards its stable points for e — » 1, as a function of payoffs (b, c). Significant information exchange at the indirect reciprocity 
behavior mode (a = l,/3 = 0) is stable for Chicken, Leader and Battle of the Sexes games (see Fig. IB). (E) Development 

of communication with time in case of fluctuating payoffs (b = 1.5, c = 0.5, V Ab 2 — V Ac 2 ~ y (AS/S) 2 ), where S is size 

of the population. Fluctuation of the payoffs (b,c), associated with the fast fluctuations in size of the population y (AS/S) 2 , 
accelerate the development of maximal information exchange at the indirect reciprocity behavior mode (a = 1, /3 = 0), changing 

the corresponding development timescale. (F) Population size fluctuations J (AS/S) 2 leading to equal development rate of 
/ (dl/dt oc Ve) at e = 0.5 for different values of payoffs (b, c). For small population size fluctuations the fastest information 
exchange rate will occur with Chicken and Battle of the Sexes games. 
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APPENDIX A: SUPPLEMENTARY MATERIALS 
FOR A. FEIGEL "CONDITIONS REQUIRED FOR 
THE EVOLUTION OF COMMUNICATION 
ABILITIES WITHIN A SPECIES" 



where p(h) is the density of individuals h in the popu- 
lation and summation goes over all individuals with dis- 
tinct properties (phenotypes). 



This supplement includes a detailed description of the 
analytical results presented in the main text. Analysis of 
evolutionary stability of populations with significant in- 
formation exchange between its members comes after an 
introduction to the evolutionary game theory. The evo- 
lutionary conditions required for the development and 
diversity of information exchange, as well as the corre- 
sponding population dynamics, are derived. 



1. Evolutionary game theory 

a. Selection rules as a game 

In the framework of evolutionary game theory, a com- 
petition of two individuals is presented as a game. For 
two-reaction games, the selection rules can be presented 
by payoff tables: 
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W mode 


W rnode 
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W rnode 


W rnode 



(Al) 



where W™ ode is the payoff for reaction p made in the 
specific behavior mode against a competitor's reaction q. 
We assume that the payoffs of an individual during an in- 
teraction depend on its mode, corresponding to different 
individual states (e.g. age, intra-population rank, sex, 
ability to acquire information and etc. . . . ) affecting the 
personal payoffs during an interaction. 

Reactions S and C can be defined as selfish and coop- 
erative correspondingly, assuming Wgg de < Wg l ^ de . 

The total gain G(m, h) of an individual m for an in- 
teraction with an individual h is: 



G(m, h) 



i,3 



>p( mi \h 3 \w;, 



(A2) 



b. Dynamics and evolutionary stability of a population 

A population can be defined as a density distribution 
over all possible phenotypes £. The density p(£), 
therefore, corresponds to the relative amount (frequency) 
of phenotypes £ in the population. Evolution of a popula- 
tion consists of two processes: redistribution between ex- 
isting phenotypes from generation to generation (natural 
selection) and emergence of new phenotypes (mutations) . 

The natural selection is described by the replicator dy- 
namics equations [46| , assuming that the fitness is a mea- 
sure of the progeny: 



where H(mj, hi) is the probability for the individual m to 
be in the mode i while its competitor h is in the mode j 
and P(mi\hj\Wp q ) is the payoff of m: 

p(m, t \h 3 \w; q ) = n ss (m i ,h j )wi !S + n sc { mi , ^)wj c + 

+Q, C s{mh h 3 )W l cs + Qccirrii, hj)W l cc , 

(A3) 

where fl pq (rrii,hj) is the probability for individual to in 
mode i to provide reaction p against reaction q of its 
competitor h in mode j. 

The fitness F(m) of an individual to is given by: 



F(m) = J2p(h)G(m,h), 



(A4) 



dp((j) 
dt 
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gen 



F 



M, 



(A5) 



where T gen is the time span of a single generation, F(£) 
is the evolutionary fitness (|A4[) of the phenotype £ and 
M is a contribution of mutations to the density p(£) of 
the phenotype £. The average fitness F in the population 
is: 



F 



(A6) 



The difference between the fitness F(£) and the average 
fitness F defines the growth (F(£) > F) or decay < 
F) of the corresponding phenotype density /?(£)• 

The evolutionary stable phenotype £ (st) has maximum 
fitness, meaning: 



(A7) 



for all x 7^ £ ( s t) ■ in the case of a population composed of 
two phenotypes (mutant and evolutionary stable host), 
the condition (IA7I) reduces to: 



(A8) 



G(host, host) > G(mut, host), 



where G(m, h) is the average payoff for an interaction 
between phenotypes to and h (see eq. (|A2|) ). 

To obtain condition (|A8[) . let us consider a mutant 
with a density (p(mut) = Ap) invading a population 
composed of some host phenotype (p(host) = 1 — Ap). 
Taking eq. (j A4[l into account, the finesses of the host and 
the mutant are: 

F(host) = (1 - Ap)G(host, host) + ApG(host, mut), 

(A9) 

F(mut) = (1 — Ap)G(mut 1 host) + ApG(mut,mut). 

(A10) 

In the limit of Ap « 1, conditions (lA7) (F(host) > 
F(mut)) and (|A8|) converge to each other. 
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In the case of equality G(host, host) = G(mut, host) 
the mutant can become evolutionary beneficial in case 
G(host,mut) < G(mut,mut). The population dynamics 
in this case is slow, depending on rare (oc Ap 2 ) mutant 
vs. mutant interactions. 



F(~r) = / d~r 'p(?')G{~?,'?'), 



F= <Trd'r'p{~r)p{~r')G{~r 1 T''), 



(A13) 



(A14) 



2. Evolution of a population as a particle-like 
motion 

In this work, whenever possible, the evolutionary dy- 
namics is going to be considered in the limit of a con- 
fined population, composed of a single phenotype with 
small variations. We are going to develop the particle- 
like equations of motion of such population in the phe- 
notype space, under assumption that the mutations (see 
[5])) correspond to a diffusion- like process: 



M = D 



d 2 t 



d' 



(All) 



where D is a diffusion coefficient and is the pheno- 
type space coordinate. For small D, mutations cause 
only minute changes in the existing phenotypes and a 
population remains, therefore, confined during its entire 
evolution. The motion of a confined population can be 
described by local velocities (a flow field) in the pheno- 
type space, similar to the motion of a particle. 

The dynamics of a population (eq. (|A5[0 . in case of a 
diffusion- like mutation process (eq. l|Alip ). is described 
by: 



dpijr) 
dt 



1 



F(~f) - F 



T„, 



-P( r ) 



D 



d 2 P 

dl^ 2 



where an instantaneous state of the evolving population 
is defined by a density distribution p(~r) over some TV- 
dimension phenotype space ~r* = (xi, X2, %n)- The 
fitness F(T*) and the average fitness F are described by 
the continuous versions of eqs. I|A4[) and (|A6[) . 

A confined population is described by its spread (size 
in the phenotype space) and by its asymmetry along the 
direction of the propagation. Its dynamics, therefore, 
can be approximated by a <5-function, taking into ac- 
count both the spread 5^ (the second derivative) and 
the asymmetry 5^ (the first derivative): 



P (r,t) = 8{?--? (t))- 



(*)) 



r (t)a< 2 >c 



where ~r*o(t) is the location of the population, ~xl 
J p(xi — Xoi) corresponds to the asymmetry and x 2 
J p(xi — xoi) 2 contributes to the spread (a 



?o(t)), 
(A15) 



x% 



The velocity of a population 



Org 
dt 



is: 



~v = v lst +v 2nd , 



(A16) 



(A12) where the first v lst and the second v lst orders are: 



Tgen G{ r , r ) 



dxi 



{4-xf) 



(A17) 



vt d (r) 
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1 
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1 d 2 G{~r, ~r') f-s -5. 



dx 2 



(4 - x 2 xi) 



2 2 ' 

x i ~ X i ) x j 



dG(?,~?') + dG(~r, ~r') 
dxidx'j dxidxj 



1 dG(~r, ~r' ) ( dG(~r, ~r') dG{~r , ~r') \ _ 



G(r , r) dx t 



dxj 



dx 'j 



(A18) 



and 1" = f p(xi — xoi) n . For derivation of eqs. (|A17[) following equations see section I A 9l In the limit of almost 

stable population, the spread is defined by diffusion in the 



and (|A18p . see section lA~8l 

The spread (^x 2 — Tf^j and asymmetry ~x~l are esti- 
mated in the limits of slow and fast propagations of the 
population in the phenotype space. For derivation of the 
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vicinity of the stable position: 



xl-x?) cx 2\l —DT qen 



{ dxl 



(A19) 



OXi \ OX 

The population remains confined under condition: 
<9 2 G(V,y) 



dx 2 



< 0. 



(A20) 



(A21) 



In case of a fast propagating population, the spread 
is equal to the diffusion during the time that takes the 
population to pass its own size: 



(xf-xf) cx {DT gen y 



dG(~r,T") 
dxi 



,(A22) 



The corresponding asymmetry of the propagating popu- 
lation is: 



x t cx (DTg, 



f dG{~r 1 T n ) 
V dxi 



(A23) 



Diversity (multi-stability) of the dynamics of a con- 
fined population requires local stable states i; = either 



dGC?,~?') 



(A24) 



or if further propagation is impossible due to the bound- 
aries of the phenotype space: 



v(rb) n < 0, 



(A25) 



where rt is a point on the boundary of the phenotype 
space and n is the normal vector to the boundary. 

In this work, a special case of: 



<9G(y,y) d 2 G(~r,T") 



dx. 



dx 2 



0. 



(A26) 



is important, in addition to the conditions (|A24[) and 
(IA25[) . The corresponding velocity v s depends entirely 
on the evolution in other directions: 



i 
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Tgen G{r , r) 



3^ s 
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dx s dx'- 



dx s dxj 



(A27) 



A population can be stable at many different points along 
s, provided that for each value of s there are stable values 
for other phenotype coordinates (see (|A24[) ). 

The equations for vj st (~r), v} at (~r), (x 2 — Xij and xl 

define the main properties of the evolutionary dynamics 
of a confined population; First, the lowest order contri- 
bution v lst to the velocity of a population corresponds 
to the deviation from stability condition: 



dG{~ 



dxi 



= 0, 



(A28) 



in case where (|A8[) holds. Second, the motion in one 
direction contributes to the velocities in other directions; 
Vj st designates asymmetry along the j direction x] ^ 

and finite contribution to v 2nd ^ for i ^ j, see eq. 
CH8]). 



3. Evolution of information exchange 



The analysis proceeds along the following steps: First, 
all possible individual strategies (the phenotype space) 
are defined. Second, a special set of the competitions (ta- 
bles of payoffs W pq ) is demonstrated to satisfy the evo- 
lutionary stability of the phenotypes, corresponding to 
the maximum information exchange. Third, mutual af- 
fective behavior during a competition and zero payoff for 
a non-responsive (stochastic) behavior are demonstrated 
to be essential for reasonable evolutionary dynamics and 
its diversity (multi-stability). The main result is that 
fluctuations of evolutionary conditions accelerate the de- 
velopment of information exchange. 
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a. Individual phenotypes 

In this work, a two-reaction phenotype is composed of 
different behavior modes k (k £ {l,N}). A specific be- 
havior mode is described by a table of payoffs W£ q (see 
(|A1[) ) and three parameters (ek,Uk, (3k)'- an individual 
possesses the probabilities to be involved in a compe- 
tition having payoffs W pq , and conditional probabilities 
afc and (3k defining its method (strategy) to choose one 
of the reactions. The behavior modes differ from each 
other by their payoffs Wp q and by the allowed values of 
a and (3. 

Parameters otk and (3k define the statistics of reactions 
of an individual in mode k against selfishness and cooper- 
ation correspondingly: otk and 1 — otk are the conditional 
probabilities to choose cooperation C and selfish S reac- 
tions against the selfish reaction S, while (3k and 1 — (3k 
are the conditional probabilities to choose cooperation C 
and selfish S reactions against the cooperation reaction 
C. 

To take into account affective behavior, additional as- 
sumptions are required: First, the parameters (e^, o^, (3k) 
depend on each other, e.g. e£(ef, af , (3f) indicates that 
individual B affects the probability of the individual A 
to be in behavior mode k. Second, Wr„ can depend on 
the probability to be in behavior mode (frequency de- 
pendent payoffs) , as in the case of an evolutionary reward 
that is divided either across many competitions or during 
a major one. The payoffs W^ q , however, are assumed to 
be independent of the probabilities a and (3. 



b. Interaction of two individuals: payoffs 



following the definitions of a and (3. 

Causality makes solution of (|A30| in a general case 
impossible; in the course of pairwise interactions, simul- 
taneous conditional reactions and information exchange 
are impossible. The second to respond, therefore, pos- 
sesses an advantage by knowing the reaction of the com- 
petitor. Exact solution, however, exists for some cases, 
turning out to be the most relevant for the evolution of 
information exchange (see section [A 6[) . 

The solution of the system (|A30|I exists for the case of 
(a.i,(3i) = (a.j,(3j) and at the boundaries of the {a.j,(3j) 
space: a^j — 0, 1 or (3ij =0,1 (see section |A 12|) . Ac- 
cording to the definition of a and (3, the solution of (|A30[) 
comes in the form: 

ttss(ei,ai,(3i\ej,aj,[3j) = (1 - 0^)7^, 
tosc(ei,ai,/3i\ej,aj,l3j) = (1 - - 7,,), 

ncc(ei,ai,Pi\ej,aj,Pj) = /3j(l - 7#), (A31) 

where 7jj is the probability of individual j to provide 
reaction S during a competition with the individual i. It 
follows from the self-consistent (mean field) conditions: 

Hi = (1 - Vjhij + C 1 - /%)(! ~ Hj), 
Hi = (1 - Oihji + (1 - (3i)(l - 7ii), (A32) 

indicating that probabilities jji and 7^ correspond to 
the same values of ti pq (m, , hj). Solving this system of 
equations results in: 



To find the evolutionary payoff (|A2j) of an individual 
to in mode i interacting with an individual h in mode j, 
one should calculate the statistics of the mutual reactions 
ti pq (rrii, hj) and mutual behavior modes S(mj, hj) during 
a competition, as function of the individual parameters 

er,«r^r and e£, 

According to the definition of e: 

E(ei,ai,f3i\ej,aj,0j) = e-jCy, (A29) 



Tj'i 



{\-(3 ] )-{l-f3. l )(a^P ) ^ 
1 - (oti - (3i)(o<j - (3j) 



(A33) 



In a homogeneous population of identical individuals 
(ei,ai,(3i), eq. (|A33|) becomes: 



l~/3 
l + a-(3' 



(A34) 



indicating that the probability of a mode i against mode j 
competition is the product of the individual probabilities 
to be in the modes i and j correspondingly. 

To derive the statistics of the mutual reactions 
ti pq (mi, hj) as a function of (cti,f3i) and {otj,(3j), one 
should solve the self-consistent system of equations: 



n 



cs 



n 



se- 



ll, 



ti C s + ^ss 
ticc 



tisc + ticc 
tiss + tisc + tics 



, ft 



tiss + tisc ' 
= ticc 

ticc + tics 
ticc = 1, 



(A30) 



where 7^ is independent of e^. For derivation of eq. 
(|A~33)) . see section 1X131 

The total gain of an individual m in an interaction 
with an individual h (see eq. (|A2|) ). becomes: 



(A35) 



G{m,h) = ^T4 p i m i\ h Aw; q ), 



1,3 



where summation goes over all behavior modes The 
payoff for a competition P(mi\hj\W pq ) (|A3[) . taking into 
account (|A31[) . becomes: 



15 



P( mi \h 3 \W; q ) = (1- a^hjiWh + (l-/3T)(l-7ii)Wj c 

I 



(A36) 



where 7^ is defined by eq. (|A33|) . 

In the general case, J7 pg can be derived as a function 
of (ai,Pi), (a.j,Pj), 7y, 7^; and additional parameters 
describing causality issues in the course of pairwise in- 
teraction. The arrangement to be the first or the second 
to respond during a competition can be defined in many 
different ways; e.g. it can be random or an individual 
with more advanced abilities to process information can 
possess an advantage. For instance, in case of equiprob- 
able right to be the second to respond, the statistics of 
the mutual reactions is: 



tics of the mutual reactions Q 



M pq {i\j) +VL pq {j\i) 



(A37) 



where Q, pq (i\j) corresponds to (IA31j) . Fortunately, the 
main conclusions of this work are independent of the spe- 
cific choice of the causality mechanism. 



c. Information exchange of two identical individuals 

An amount of information must be transferred between 
two interacting strangers to allow correlated, rather than 
random, mutual reactions. Information exchange be- 
tween two interacting individuals follows from the statis- 



J 



pq ■ 



p,q=S,G 



n pq l0g 2 



On 



(QpS + Qpc)(QSq + ftCq) 



(A38) 



under assumption that no information is shared prior to 
the interaction. 

To derive information exchange per behavior mode, 
one should substitute eqs. (|A34p and (|A31|) into eq. 
(|A38jl . assuming that the individuals compete in the 
same mode (ai,(3i) = (ctj,j3j) = (a,/?). The correspond- 
ing statistics of mutual reactions (I A31|) is: 



n S s = (l-a)7, fic<7 = 0(1-7) 

n cs = aj, n sc = (i-/3)(i-7), 



where, according to eq. (IA34[) 7 is: 

1-/9 
1 + a-P' 



7 



(A39) 



(A40) 



and index i is omitted as unnecessary in this case. 

Substitution of eqs. (|A39[) and (|A40j) into eq. (|A38|) 
results in: 



(i - «)7io g2 ['£^1 + (i - m - 7) io g2 ({1 - m - 7) 



7(1 - 7) 



-«7log 2 



(1 - 7)7 



(A41) 



r 



taking into account that for p,q = S: 



Ci p s + Q p c — &Sq + Qcq — 1, 



nipulations to: 



and, in case of p, q = C 



fl pS + fl p c — &Sq + ^Cq = 1—7; 



(A42) 



(A43) 



since these summations correspond to the unconditional 
probabilities to provide either selfish or cooperative reac- 
tions (see Fig. 2B). 

The expression (IA41[) can be reduced by symbolic ma- 



log 2 



(1 - a)^- a ^(l - /3)(l-/5)( 1 -7) a «7 /f3 /3(l-7) 

7 7 (l-7) 7 



(A44) 

where 7 corresponds to (|A40|) . 

The main properties of information exchange (|A44[) are 
presented in Fig. 2C. I varies from to 1 bit, describ- 
ing a binary reaction. The maximal transfer (1 bit per 
interaction) corresponds to the indirect reciprocity (IR) 
(a = 1,(3 = 0) and synchronous (a = 0, (3 = 1) behav- 
ior modes. No information transfer is required in cases 
of random reactions (a = (3), homogeneous cooperation 
((3=1) and selfishness (a = 0). 
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d. Instability of the indirect reciprocity and extremely slow 
development of synchronization in populations with a single 
behavior mode 

The main result of this section is that at least two be- 
havior modes are required to evolve significant informa- 
tion exchange. In populations composed of individuals 
with a single behavior mode (a,/3), indirect reciprocity 
(a = l,/3 = 0) is always unstable and an infinite time is 
required to develop the synchronization (a = 0, = 1). 

A population composed of a host with a single indi- 
rect reciprocity behavior mode (a = 1,0 = 0), is al- 
ways unstable either toward a (a = 1 — Act, = 0) or 
(a = 1, = A/3) mutants. An individual (c*i = 1, 0i = 0) 
expresses unconditional selfishness jji = or coopera- 
tion jji = 1 against mutants (oj = 1 — Aa, 0j = 0) 
and (cxj — l,0j — A/3) (Aa, A/3 <C 0) correspondingly, 
though possesses ja = 0.5 for interaction with an identi- 
cal individual: 



7ji(a = 1,(3 = 0|a 



l,/3 = 0)= 7 »(l,0) = i (A45) 



lim 7,-j(a 

Aa^O 



1, /3 = 0|a = 1 - Aa, = 0) = (A46) 



A lim o 7 Jl (a = l,/3 = 0|a= 1,0 = Afi) = 1, (A47) 

The corresponding payoffs (|A3[) are (taking into account 
eqs. (IA45l . (rA46l) . (TAT7ll and SMTP : 



G(h, h) = (Wsc + Wcs)/2, 



G(m Q , /i) = W S c, 



G(m ,h)=W CS , 



(A48) 



(A49) 



(A50) 



e. Stability and diversity of indirect reciprocity 

Consider a population composed of individuals, pos- 
sessing several behavior modes (e/c,afc,/3fc) (k = 1..N). 
The main goal is to find the evolutionary conditions 
(payoffs Wp q , mechanisms of affective behavior and con- 
straints on the exchange of information) required for 
the development and stability of the indirect reciprocity 
mode (am = 1, 0m = 0). An additional requirement 
is the diversity of the evolutionary dynamics, meaning 
that multiple stable states of a population exist under 
the same evolutionary conditions. 

We will demonstrate that stability of the indirect reci- 
procity requires greater evolutionary payoff for the less 
frequent pairwise interactions. The individual payoff 
Wl? for a pairwise interaction, must be inversely pro- 
portional to em, the probability of the individual to be 
found in the indirect reciprocity mode: 



W IR ot 

pq 



l 



In this case, the table of payoffs (|A1[) is: 



tmode 





c 


S 


c 


W mode 


W rnode 


s 




W rnode 



(A53) 



(A54) 



where W™ ode corresponds to the possible evolutionary 
payoff for a such behavioral mode. 

Condition (| A53|) is essential for evolutionary stabil- 
ity of the indirect reciprocity mode (cq = l,0x = 
0) in a population (e^ a k^k) against either (a™ = 
1 - Aa,/3f = 0) or (af = l,^] 71 = A/3) mutants, 
Aa, A/3 -c 1. The corresponding conditions of the evolu- 
tionary stability (|A8[) are (see section |A 10|) . for the case 
of (a\ n = 1 - Aa,0f = 0): 



One of the mutants, therefore, possesses a payoff greater 
than the host, since either: 



Wlr-, 

— -SO- + AaeiB > 0, (A55) 



W sc > (W sc + Wcs)/2, 



(A51) and for the case (af = 1,0? 



-A/3): 



W cs > (Wsc + W cs )/2. 



(A52) 



Consequently, indirect reciprocity can not develop in 
populations with a single behavior mode, since the con- 
dition (|A8[) is always broken. 

The stability of synchronous population is ill-defined: 
it can be stable, though requires infinite time to develop. 
Populations with a = are identical with homogeneous 
reactions S vs. S, while populations with 0=1 are iden- 
tical with homogeneous reactions C vs. C. The dynamics 
near these axes are very slow, since there is no evolution- 
ary benefit for transfer between two identical states. A 
population, therefore, can be stacked in the vicinity of 
the synchronous mode (a = 0,0 = 1) but is unable to 
reach it. 



Whs 



A0e 1 D > 0, 



(A56) 



under assumption that = e" 1 = e% and keeping only 
the first order terms of Aa and A/3. Inequalities (|A55[) 
and (|A56[) can not hold simultaneously without condition 
(IA53|) ; the terms oc e\ are dominant and possess opposite 
signs. 

Under condition (|A53|) . the expressions (|A55[) and 
(|A56[) become: 



ei 



cs - — + AaB > 0, 



W sc 



Whs 



A0D > 0, 



(A57) 



(A58) 
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taking into account that both B and D are proportional 
to W£ q (see section IXTU|) . In the case e-y -> 0, (|A57|) 

and (|A58[) converge to AaB and A/3Z) correspondingly. 
The stability of the indirect reciprocity behavior mode is, 
therefore, possible when the probability to be in such a 
mode is low (em — > 0, am = 1, /3/^ = 0) and B,D > 0. 
A scaling of the payoffs W pq oc e A , A ^ — 1 (different from 
(|A53|) ) either prevents stability of the indirect reciprocity 
mode (A > —1) or makes the expression for evolutionary 
gain (|A35P to diverge (A < —1). 

Stability analysis of the indirect reciprocity mode 
(em — > 0, am — 1, = 0) against (ei = 1 — Ae, a™ = 
1, = 0) mutants, brings no significant confinements on 
the required evolutionary conditions; this mode is stable 
for changes in e under quite general choice of payoffs. 

Affective behavior is required to allow for the develop- 
ment of the indirect reciprocity behavior mode (em — > 
0,am = 1,Pir = 0). The condition (|A53|) suggests two 
different interpretations of the parameter ej, either as an 
individual probability to be in mode i or as a probability 
to affect the competitor to behave in mode i. The cor- 
responding payoff for an individual (««, ei) competing 



against (ay , /3j , e,-) is: 

G(A ) B)=53cfP(a^,/^ 



|n T ,/?f|T4^), (A59) 



or, in case of affective behavior: 



G(A,B) = J2etP(<xf, 



(A60) 



In the limit of e h — e m these interpretations are indis- 
tinguishable. The development of em — > 0, required for 
the stability of the indirect reciprocity, is impossible in 
case of (|A59[) : v e = 0, since derivatives for any order n 
vanish, d n G(7t, T])/de r > = (see (lATTl) andjMH])). Con- 
sequently, affective behavior (|A60[) is essential to ensure 
evolutionary dynamics of a confined population subjected 
to condition (|A53p . 

Diversity requires the existence of at least two different 
modes (e.g. mode 1 and mode 2), that in interaction with 
an arbitrary mode i will provide an equal payoff: 

P(a i ,pi\a 1 ,lh\W*)=P(a i) fr\a2,lh\WL). (A61) 



The multi-stability along e\ direction requires: 
dG(m, h) 



de^ 



0, 



(A62) 



taking into account (|A62[) and assuming W* q to be inde- 
pendent for different i. The modes 1 and 2 have to be 
linked by some permanent constraint to ensure condition 
Ml) . 

The condition for diversity (|A61[) holds if for any i, ei- 
ther 7xi = 72i (assuming the payoffs P defined by (|A36p ) 
or Wp q — 0. For instance, in case of only two behav- 
ior modes: responsive (R) and non-responsive (NR) , the 
non-responsive mode may correspond to stochastic be- 
havior with reaction statistics 77?, matching the reaction 
statistics of the individual in mode (R) : 



leading to: 



otNR = Pnr = 1 — 7-R) 



P(a R ,f3 R \a R ,(3 R \WK) 



-P(a R ,(3 R \a NR ,(3 NR \W*) = 0. 
Zero payoff for non-responsive behavior: 



W NR = 

yv pq yj - 



is required to ensure: 



P(a NR ,(3 NR \a R ,(3 R \W™)-... 
-P(a NR , [3 NR \a NR , /3 NR \W* R ) = 0, 



(A64) 

(A65) 
(A66) 

(A67) 



since the payoffs can not be the same for an interactions 
with (responsive vs. non-responsive) and without (non- 
responsive vs. non-responsive) information transfer. Di- 
versity is preserved in the case of many behavior modes 
under condition that an individual's non-responsive be- 
havior depends on the competitor's mode i, matching the 
average statistics of reactions against this specific behav- 
ior mode. 

The condition (|A64[) possesses a reasonable interpreta- 
tion and brings no additional parameters to the model, 
contrary to other possibilities to explain the required in- 
distinguishability of the two different modes (|A61[) . The 
non-responsive behavior can be associated with reactions 
induced by an arbitrary member of the population, rather 
than by the opponent. In this case the probability of a 
specific reaction matches the average reaction statistics 
of the individual. 



Evolution of information exchange in a 
population with two behavior modes 



Let us consider an example of a population composed 
of individuals with two behavior modes: non-responsive 
(R) and responsive (NR), and subjected to the condi- 
tions (|A53j) . (|A61j) and (|A64|) . The corresponding indi- 
vidual phenotype consists of three parameters: selfish- 
. . ness aversion a, cooperation attraction (3 and ability of 

' = ^(F(a l ,A|ai,^i|W; g )-P(a 4 ,/3 l |a 2 ,/3 2 |^; (; )),affect e. In the responsive mode, a and 1 - a are the 
; probabilities to choose cooperation C and selfish S reac- 

(A63) tions against the competitor's selfish reaction S, while j3 



for multiple values of e\. Diversity along a and (3 direc- 
tions does not occur unless the weights W pq depend on 
a and (3. Condition (|A61j) follows from eq. (|A60j) and 

£ 2 = f -Ejy 2 e J : 



18 



and 1 — are the probabilities to choose cooperation C 
and selfish S reactions against the competitor's coopera- 
tion reaction C. In the non-responsive mode it generates 
selfish and cooperative reactions with probabilities 7 and 
1 — 7 correspondingly, where 7 is the individual probabil- 
ity to be in state S averaged over all (both non-responsive 
and responsive) interactions. Ability of affect e is the 
probability to put a competitor into its non-responsive 
mode. 

The evolutionary payoffs for each behavior mode are 
described by the tables of payoffs (|A1[) . There is no pay- 
off for non-responsive behavior W NR = 0. The weight 
table for responsive mode W R can be reduced to a two 
parameter form (see section [A 11|) : 





c 


S 


c 


1 


c 


s 


6 






(A68) 



when the average fitness in the population is: 

Fal, (A69) 

This condition corresponds to a population that is almost 
stable in size, since the fitness is a measure of the progeny. 

The fluctuations of payoffs b and c correspond to the 
short time fluctuations in the size of a population S: 



Ac 2 , A6 2 oc l^—j . (A70) 

In case of non-zero sum game the population size changes 
with time, and the fluctuations are, therefore, relative to 
the average growth rate of the population. 



a. Dynamics 

This section demonstrates that development of infor- 
mation exchange requires payoffs b and c to fluctuate. 
Otherwise, the population converges to one of its stable 
points with affect ability e ^ 0, far from the indirect reci- 
procity mode. The motion of the population, as a con- 
sequence of these fluctuations, makes the development of 
the affect ability e — ► 1 and stability of the indirect reci- 
procity mode (em — > 1, ajjj = 1,&ir = 0) possible. The 
evolution of information exchange occurs in the vicinity 
of the stable points, in case the fluctuations of b and c 
are small. 

Consider a population composed of phenotypes 
(ej, ati, fa), with corresponding densities pi(ei, at, 0i). 
The payoff (|A60[) for an individual i interacting with an 
individual j is: 

G(i,j) = (l-e i )P(a i ,A|7ii|W^) + 

+e l P(a l ,fc\ l3 \W; q ), (A71) 

where 7^ (|A33p and jj are the probabilities of individ- 
ual j to provide reaction S in its responsive and non- 
responsive behavior modes correspondingly. The payoff 
P according to eq. (|A36|) becomes: 

P(a,/3| 7 |W M ) = a 7 c+ (1 - /3) (1- 7)6 +/3(1- 7), (A72) 

taking (|A68[) into account. 

The probability 7, follows from the self-consistent sys- 
tem of equations similar to eqs. (|A32[) : 



7l - K 1 - e i)^(( 1 ~ + (1 ~ ~ In)) + ~ atihj + (1 - - 7,)] . 

j 

(A73) 



averaging individual probability to be in state S over 
all pairwise interactions. In a homogeneous population 
(e,a,0): 



lj = Iji 



l + a-0' 



(A74) 



indicating that the non-responsive and responsive behav- 
iors are indistinguishable from each other. For derivation 
of eq. (|A74|I . see section IA131 

The motion of a confined population is derived by sub- 
stituting the individual gain (|A71[) into the equations for 
the velocity of the population (|A17|) and (|A27|> . The first 



order v] st vanishes: 



vl s \e,a,0) ex P{a i ,0 i \ lj \W' pq ) - P(a t , fch^W^) = 



(A75) 



since jj — jji in this limit (|A74I) . The local velocity of a 

■ Is' ' 

J 



population 1? = (v^ st , vl st , v^ nd ), is defined by: 



^ st (e,a,/3) 



1 



T, 



gen 

mm 

dai 



dP(i\j) djjj 



(A76) 
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^ st (e,a,/?) 



1 



T,„ 



(xj -xp)x 



dv\\ 
dxn 



< 0, 



(A80) 



-mm 



+ (l-ei) 



mM djii 



(A77) 



2nd / o\ 



1 _ 2 ^|j) 



■(a^-ST) 



^7ji , din 



dcti 



daj 

9 7: 



ji 



OP, 



(A78) 



where all derivatives are taken at the point (e, a, j3) — 
(ei,ai,Pi) = (ej,ctj, Pj). The velocity is proportional to 

the spreads (x^—xl) and to asymmetries x~p (k — e, a, /3) 
of the population (see section [A~2j) . For derivation of eqs. 
(TA75]) . jMH, (jATf)) and (TA78]) . see se ction j XTj 

The payoff P{i\j) is defined by cq. (|A36[) only at the 
boundaries (a — 0, 1) or (/3 = 0, 1). In other regions of 
(a, p) the space eqs. (|A31[) are invalid; to calculate the 
statistics of the mutual reactions £l pq , the exact order 
of reactions in a course of the interaction must be taken 
into account, e.g see (|A37j) . The velocities (v a , vp, v e ) for 
arbitrary (a, /3) are calculated by substituting the payoff 
P(i\ j) in its general form (fA"3|) into eqs. (|A17|) and (|A27|) . 



v±n < 0, 



(A81) 



where (v\\,v±) are the components of the velocity par- 
allel and perpendicular to the boundary, while n is the 
normal vector at the boundary. The position of a stable 
point (a st ,/3 st ), defined by conditions (|A79[) and (|A80[) . 
is independent of the order of reactions in the course of 
an interaction, since there exists a solution for (| A31|) at 
the boundary of the phenotype space. To analyze (|A81[) 
the equiprobable order of reactions is assumed (|A37[) . 

The evolutionary stable points for payoffs b > and 
b > c, following (|A79[1 and (|A76[) , are (see section [Xl5)) : 



-(1 - e) + v /(i-e) 2 +4eif 2 



2eK 



(A82) 



5. Evolutionary stable states on the boundaries 



A confined population on the boundary of the (e, a, (3) 
space is evolutionary stable in case of: 



0. 



(A79) 



where K = c/b. 

For payoffs b > 1 and b < c one obtains, following 
(|A79|) and (|A77|) . the stable points: 



a st = 0, 

Pst = 



-(-2 + (1 - e)(l - Z)) - v /(-2 + (l-e)(l-Z))2-4e(l-e)Z 



2c 



(A83) 



where Z = (c - l)/(6 - 1) - 1. 

For other payoffs (6, c), a population is stable at some 
arbitrary point on cither a = or p = 1 plane. The 
evolutionary dynamics in the vicinity of these planes is 
slow and the stability position is undefined, since |f || | = 
holds for all points. 

In case of developed affect abilities e — > 1, the indirect 
reciprocity behavior mode is stable for the region (b > 
l,c > 0), including Chicken, Leader and Battle of the 
Sexes games. Prisoner's Dilemma degrades information 



exchange. These results are illustrated in Fig. 4. 



6. Games favoring development and degradation of 
information exchange. 

The individual ability to exchange information in- 
creases with the development of affect ability e. Stability 
of the indirect reciprocity behavior mode is achieved only 
at e — > 1. Consequently, the evolutionary conditions fa- 
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voring and degrading information exchange correspond 
to the positive and negative values of v e (|A78[) . 

Small fluctuations of the payoffs b and c induce motion 
in the vicinity of the stable points, with the average vl oc 

1 d 2 v e (e, a, P, b, c) 



dc 2 

1 d 2 v t (e, a, (3, b, c) 
2~ 



Ac 2 + 



a st ,0 st ,b,c 



db 2 



Ab 2 , (A84) 



otat,/3 a t,b,c 



under assumption that Ab, Ac = 0. In case of constant 
payoffs b and c, a population converges to a stable point 
with v t = (see §K7ty ). 

Information exchange is favored (vl > 0) in the 
Chicken, Battle of the Sexes and Leader games ((b > 
0, c < b) and (b > 1 , c > b)). The evolutionary dynamics, 
therefore, occurs in the vicinity of the evolutionary stable 
points on the axis = or a = 1. For the (b > 0, c < b) 
region, the development rate vl follows from eqs. (|A84j) 
and l(A78|) : 



v e (e,a,0) 



Ap 2 



2T, 



■(x 2 



2 ) 



yen 



dc 2 



Q2J 

db 2 



dG{i,j) (d 2 G(i,j)\ 1 dP(i\j) fdjji , d l3 



da* 



da 2 



dot; 



dotj 



(A85) 



and for (b > l,c > b) one obtains: 



/W2 

w7(e, a, P) = -— — {x 2 - xf) x 

A-*- gen 

8G(i,j) (d 2 G{i, ] )Y 1 8P(i\ 3 ) (d l3l , a 7j 



dp 2 



(A86) 



where Ap 2 ps Ab 2 , Ac 2 . The asymmetries of a population 
are estimated along the boundaries according to (|A20j) 

and in the perpendicular direction according to (|A23p . 

The asymmetries x± are neglected since (DT gen )3 <^ 1, 

in the limit of small D. 

The size of the of the population (x 2 — ~xl 2 ) along the 

e direction is estimated as a diffusion distance during a 

time Tk that takes for a population to pass its own size 

along the a or P directions. It results in: 



(x 2 - x e 2 ) cx (DT gen 



\ dkdc J \ dkdb J 

(A87) 

where index k is either a or p. This equation fol- 
lows from (x 2 — x^ 2 ) oc DT^, taking into account Tj, = 

{xj, — Xk 2 )/\vk\, where \vk\ is the absolute average ve- 
locity of a population subjected to fluctuating payoffs 
(b, c) near its stable point: 



T ge 

d_ d_ 

dc db 



x\ Xi \ X 



dG{~r 



dxi 



(A88) 



see eq. (|A17|) . 

The contour plot of payoff fluctuations Ap 2 causing 
equal development rate (|A86[) at different points of 
the payoff space (6, c) (with e = 0.5) is shown in Fig. 4F. 



7. Information exchange vs. time 

The development rate of information exchange in- 
creases with fluctuations of payoffs b and c, co rrespond- 
ing to fast fluctuations of the population size (AS/S) 2 . 
Maximum information exchange I in a population de- 
pends on the affect ability e, together with the payoffs b 
and c. Consequently, the instantaneous development of 
the information exchange is: 



91 dI ~( R h \ 



(A89) 



under assumption that the average values of b and c are 
constant. Taking into account eqs. (|A85[) . (|A86[) and 
(IA87|) . one can write: 



dI 1 tl t, ^ 

^° C ^ /(e ' 6 ' C) ' 



(A90) 



where / specifies the population under discussion. The 
time of development of information exchange and pro- 
cessing T dev : 



'AS' 



2\ 2 



(A91) 



is inversely proportional to the diffusion coefficient D and 



to population size fluctuations y (AS/S) 2 , and propor- 
tional to the time-span of a single generation T gen . 



8. Derivation of eqs. (|A17[) and (|A18|) 



We are looking for a J-function like solution (|A15|) of 
eq. (IA12P . In a moving frame of reference ( r <— r — 
vt,t t), eq. (|A12[) becomes: 



,dp(~r) dp(r) 



1 



d r 



dt 



■p(~r 



' \F\ 



D 



d 2 P 



d r 2 
(A92) 

where v is the velocity of the frame of reference. 

To derive the ?th component of the velocity Vi, one 
should multiply eq. (|A92[) by the orders of X{ (where 
~x = ~r' — T^o) and integrate over the entire phenotype 
space r . The equations corresponding to the orders from 
a;? to xf are: 
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(A93) 



(A94) 



(A95) 



„ dxf 
3^f + ^ 



6L>Xj + — x? 



T, 



gen 



T F 



dG(V,~r r ) ^ dG(~t,V)—^ 

IT XkX r + ITi Lx k x i + 

l dG(~r,T ')- 
2 dxkdxj 



-X k XjX^ 



ldGCr,-?') 
2 dx k dx\ 



j-^t 1 dG{ r , r ') , , ,, 



( A96) 



The velocities of a confined population (eqs. (jATTJ and 
(|A18[) ) follow from eqs. (j A93II A96]) , under assumptions: 



to the average fitness F: 



dxi _ dxf _ dxf _ 
~dt ~ ~dT ~ ~dT ~ ' 



(A97) 



that the population is static in the moving frame of ref- 
erence. 

To derive the components of the velocity vj st (|A17|) 
and vf nd (|AT8l) . one s hould multiply eq. (|A^3)) by xi and 
subtract it from eq. (|A94j) . The other terms vanish, e.g 
x k Xj — x k Xj = unless k — j. The last term of the ex- 
pression for vf nd (|A18|) is a consequence of contributions 



F = G(~r,~r 



dG{~V,T")_ dG('r,'r')— r 



dxi 



-Xi + 



dx' 



X:, 



(A98) 



by the asymmetry of the population shape, Xi, x' i 7^ 0. 
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9. Derivation of eqs . (|A19p , (|A20|) . (|A22|) and 
(TA23]) 



At slow propagation, the spread of a population 
(a;f — x? 2 ^ (|A19|) follows from eq. (|A95[) . after subtrac- 
tion of eq. (|A93|) multiplied by x\: 



where the population was assumed to be near its stable 
position: 



dGC?,?') 
dxi 



d 2 G(T > ,T >/ ) 



dxi 



DT < 



<9 2 G(7*,^') 



dxi 



(A101) 



during an amount of time: 



(A108) 



required for a population to pass its own size. The spread 
as a function of D and derivatives of the gain P follows 
from (|A107|) . taking into account eqs. (|A17|) and (|A108|) : 



(x 2 - x 2 ^j oc 



D 2 



(A100) where velocity Vf is: 



/ m n i,„,! / dG( r , r ') 
v f = (T gen )-s(D)3 1 



(A109) 



(A110) 



V 9xf 

Asymmetry (|A23[) is estimated from eq. (|A95|) . as D/vi 



d 2 G(T>,T") 
dxi 



< 0, 



(A102) 



and Vi was neglected. In case of a ^-function like p( r ), 
condition: 



(x 2 - x 2 ^ oc (xf - xf^j 



(A103) 



is assumed to hold. 

The asymmetry (|A20|) is derived by substituting D (eq. 
C&99J) ) and vj st (eq. l|AT7| ) into eq. (|A96)l . after sub- 
traction of (eq. (|A93[) multiplied by 



,3. 



dG(T',T") fd 2 G(~r,T") 



dxi 



dx 2 



3 x ' Xj 



-2xf + |x 2 ' 



(A104) 



where terms higher than xj where neglected. Eq. (|A104[> 
corresponds to eq. (|A20|) . taking into account: 



3x 2 



-2xf + %x 2 ' 



2 i 



const < 0, 



(A105) 



in the limit of a <5-function like p(r ). 

The spread (|A22[) in case of fast propagation: 



dG(T > ,V) 

dx,. 



» 



d 2 G(~r,T") 



dx 2 



(A106) 



is estimated as the diffusion radius: 



Xi oc (DTg 



i fdG(~r,T") 



dxi 



(Alll) 



taking into account (|A106|) . 



10. Derivation of eqs. (fA55l) and (fA56l) . 



To derive eq. (|A55[) . consider a population com- 
posed of a host h and a mutant m, with the phenotypes 
(e^,a£,/3f) and (e™,a™,/?™) correspondingly. The con- 
dition (|A55[) follows from the stability condition (|A8[) : 



G{h,h)-G{m,h) > 0, 



(A112) 



under assumption that only the host possesses a devel- 
oped indirect reciprocity mode [cc[ = 1,(3^ = 0), while 
the mutant is different from the host by its aggression 
aversion (a™ = 1 — Aa, /3f = 0). All other behavior 
modes are identical: 



(A113) 



for all i 7^ 1. 

The condition for evolutionary stability (|A112[) be- 
comes: 



i,3 



dCj (PihilhjlW^) ~ P( m% \h 3 \w; q )) > 0,(A114) 



taking into account the payoffs (|A35|) . The terms with 
i ^ 1 and j ^ 1 vanish due to (|A113[) . The first term of 
(|A55[) corresponds to i = j = 1 terms of (|A114[) : 

e 2 (P(a = 1,(3 = 0\a = l,/3 = 0\W^ q ) - . . . 
-P(a = 1 - Aa,/3 = 0|a = l,j3 = 0\W^)) = ... 
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in the limit of Aa <C 1. The difference of the payoffs 
in eq. (|A1 15|) converges to a finite value in the limit 
Aa — > 1 as consequence of the instability of the behavior 
in the indirect reciprocity mode, see eqs. (|A45IIA52|) . The 
second term of (|A55[) follows from: 

^ {P(<* = 1,13 = ok-, 0j\w p \) 

-P(a = 1-Aa,0 = 0|a,,/3,|W^)) cx Cl Aa. 

(A116) 

This sum is proportional to Aa, since there are no insta- 
bilities in behavior for any (aj,0j) ^ (1,0) interacting 
with the modes (a = 1, = 0) and (a = 1 — Aa, = 0). 

The condition (IA56[) follows in the same way as (|A55I) 
assuming (a™ = 1,(3™ = A/3), rather than the (a™ = 
1 - Aa,/?] 71 = 0), mutant. 

11. Derivation of eq. (|A68j) . 



For an average fitness (|A6|) F w f, two independent 
parameters are sufficient to describe the payoffs for re- 
sponsive behavior in a population composed of individ- 
uals with two behavior modes. Reduction of the payoff 
(|Atp to a two parameter form introduces only a new time 
scale in the equation of motion (|A5|) . without having any 
impact on its evolutionary dynamics. 

To derive the table (1A68|) from (1AI j) . two transforma- 
tions are required: 



W 



W, 



I"! 



W SS , 



(A117) 



W pq = w; q /(W cc - W SS )- (A118) 
Consequently, the parameters b and c in (|A68[) are: 
W SC ~ W S s 



W cc - W ss ' 
W C s - W S s 



(A119) 



Wcc - Wss 

The transformation (|A1 18[) applied to eq. f (|A5|l ) results 



MO 
dt 



■p(0 



?m4zim + M, (Am) 



■gen 

This follows from: 



F(W pq ) = F(W; q ) + W SS , 

F{W pq ) = F(W' pq ) + Wss, (A121) 

due to the property of the payoff (|A3|) : 

PirmlhjlWpg) = P(mi\hj\W; q ) + Wss, (A122) 

taking into account that ^2 pq Qpq = 1- 



The second transformation (|A1 18|) brings eq. (|A120[) 
to the form: 



1 



dp(Q _ 
dt' T„ 



■P(0 (mw;' q ) - F(w;' q )) + M, (Af 23) 



where: 



and 



t' = t 



Wcc - Wss 
F 



7" 

pq 
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(A124) 



(A125) 



corresponding to eq. (|A68|) . 

12. Solution of the system of eqs. (|A30[) . 

The system of eqs. (|A30[) can be presented as: 

1 




\lj 

(A126) 

A solution for this system exists if the determinant of the 
corresponding extended matrix U vanishes: 
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where 
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(A128) 



For a general case there is no solution, since it includes 
five equations and only four variables. 
The determinant of U: 

detU(ai,0i,a 2 ,02) = 

ai(0i - 1)02 + a 2 {a 1 2 - 0i{on + 2 - 1)). 

(A129) 

vanishes either in case of identical phenotypes (a±,0i) = 
(a 2 ,02) or at the boundaries of the (a, 0) space, a = 0, f 
or = 0, 1. It can be demonstrated as follows: 

detU(a,0,a,0) = 0, (AI30) 

Then taking into account (|A130|) . a 2 ^ ai fits (|A127|I if: 

<9det£7 



da2 



= 0(1-0), 



(A131) 



ai,2=a,/3l,2=/3 
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vanishes. The same holds for changes in 02- 
ddetU 



take into account that: 



ai,2=Q,/3i, 2 =/3 



a(l-a). (A132) dG(i\j) dP{i\j) 



The solution (1A31|) is valid in these cases and can be 
checked by a substitution. 



doti 



and 



+ (1 - £i) 



^1^, (A136) 



13. Derivation of eqs. (|A34[) and (|A74[) . 

In a homogenous population composed of individuals 
(e, a, (3), the average probability "{nr. of the selfish reac- 
tion S is: 



Inr = e 7JV-R + e (! - e)7w_R + • 
+ (1 - e)ej RtNR + (1 - e) 2 7ii,- 



(A133) 



where one averages over all possible interactions: non- 
responsive vs. non-responsive, non-responsive vs. re- 
sponsive, responsive vs. non-responsive and responsive 
vs. responsive. The probabilities to provide reaction 
S while interacting with non-responsive and responsive 
competitors are: 



1r,nr = (1 - a)lNR + (1 - /3)(1 - inr), (A134) 



and 



1R,R 



1-/3 
l + a-/3' 



(A135) 



correspondingly. 

Solution of eq. (|A133|) for ^nr, results in eq. (|A74[) . 



14. Derivation of eqs. (|A75]) . (|A76ll . (|A77|> and 
(TA781) . 

These equations are derived substituting the gain 
G(i,j) (|A7ip in expressions (|A17[) and (|A27|) . One must 



d 2 G(i\j) _ dP(i\j) fry, 
de l da i djji da, 



ji 



(A137) 



together with similar expressions for dG{i\j) / d(3i and 
8 2 G(i\j)/dem- 



15. Derivation of eqs. (|A82[) and (|A83[) . 



To derive eq. (|A82[) . following condition (j A79|l . one 
should find the points of zero velocity (|A76[) on the f3 = 



axis: 



mm 



(1 



dP(i\j) d l3 
djjt daj 



0. 



(A138) 



Solving this equation for a results in (j A82|l . taking into 
account the expressions for payoff P (|A72[) and jji (| A33|) . 



Derivation of eq. (|A83j) is analogous to the derivation 
of eq. (|A82|) . One should substitute eqs. (|A72j) and 
(|A33[) into (|A77[) . finding its zeros on the a — 1 axis. 



The stability condition (|A80[) holds for the stable 
points (|A82|) and (|A83|) in cases of (b > 0, c < 6) and 
(6 > l,c > b) correspondingly. This can be checked by 
substituting eqs. (|A71|) and (|A72|) into condition (|A21|) 
(matching (|A80[) in case of a confined population): 



gv (l+b)/f 3 (-l+c)£ 3 (l+(-l+2if 2 ) £ - % /l+(-2+4if 2 )e+e 2 ) (-l+e+^/l+(-2+4if 2 )e+e 2 ) 

9 " (-l+e-2ife+ % /l+(-2+4K 2 )e+ e 2 ) 2 (-l+ e +2ife+^l+(-2+4K 2 )e+ e 2 ) i! ' 

iv=f, (A139) 



and 



dp 
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6(-l+£)£ 3 (/f(-l+ £ ) + ^/if 2 (-l+e) 2 +4e) (if 2 (- l+e^e+if y/ K 2 (- l+£) 2 +4e) 
(if(-l+e)-2e + ^/K 2 (-l+e) 2 +4e) 2 (_R'(-l+ e )+2£+\/K 2 (-l+ e ) 2 +4 e ) :i 
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(A140) 
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These expressions are negative in the regions (b > 0, c < 
b) and (b > 1, c > b) correspondingly. 

T he con dition (|A81j) holds for the stable points (|A82j) 
and (|A83[) in the cases of (b > 0, c < 6) and (6 > 1, c > 6) 
correspondingly, assuming equal probabilities to be the 
second to respond (see (|A37|) ) in the course of an in- 



teraction. It can be checked by substituting the general 
expression for the payoffs P (|A3p together with eq. (|A37[) 
in eqs. ([A7i~|) . (|A76l) and ||A77|) . The derived expression 
for v± are too cumbersome to be present here, though 
they can be handled with the help of a symbolic calcula- 
tion software, e.g. Wolfram Mathematica. 



